INTELLIGENT TUTORING SYSTEMS, PEDAGOGICAL AGENT DESIGN, AND HISPANIC ENGLISH LANGUAGE LEARNERS

According to the most recent data from the National Center of Education Statistics (NCES) there were approximately 5 million English Language Learners (ELLs) in the U.S. public schools in the Fall of 2016, representing about 10% of the student population (2019). Spanish is the primary language for most ELL students, by a large margin. As a group, ELLs have faced a deeply rooted and persistent math achievement gap (U.S. Department of Education, 2015). Despite research indicating that intelligent tutors and animated pedagogical agents enhance learning, many tutors are not designed with ELLs in mind. As a result, Hispanic ELL students may experience difficulty accessing the relevant content when using a tutor. This mixed-method research investigates how a tutor can reach Hispanic ELL students, based on the social and cultural Identity framework of the Figured Worlds Theory by Holland et al., (1998). Students will socially and culturally engage with their animated pedagogical agents constructing figured worlds of learning and connection that have the power to shape the students’ senses of themselves as learners of math. This study investigates how Hispanic ELL students perceive the utility of and relate to a learning companion (LC) design. Data was examined from 76 middle school students interacting with a math tutor, MathSpring. The findings indicate that ELL students find the MathSpring LC more useful and helpful than do non-ELL students and the ELL students designed LCs that looked more like themselves than did the non-ELL students. The findings also indicate that students formed ‘She/Me Connection’ and ‘She is Like Me’ figured worlds

FACE READERS: The Frontier of Computer Vision and Math Learning

The future of AI-assisted individualized learning includes computer vision to inform intelligent tutors and teachers about student affect, motivation and performance. Facial expression recognition is essential in recognizing subtle differences when students ask for hints or fail to solve problems. Facial features and classification labels enable intelligent tutors to predict students’ performance and recommend activities. Videos can capture students’ faces and model their effort and progress; machine learning classifiers can support intelligent tutors to provide interventions. One goal of this research is to support deep dives by teachers to identify students’ individual needs through facial expression and to provide immediate feedback. Another goal is to develop data-directed education to gauge students’ pre-existing knowledge and analyze real-time data that will engage both teachers and students in more individualized and precision teaching and learning. This paper identifies three phases in the process of recognizing and predicting student progress based on analyzing facial features: Phase I: Collecting datasets and identifying salient labels for facial features and student attention/engagement; Phase II: Building and training deep learning models of facial features; and Phase III: Predicting student problem-solving outcome. © 2023 Copyright for this paper by its authors

ATL-BP: a student engagement dataset and model for affect transfer learning for behavior prediction

IIS-1551572 - National Science Foundationhttp://10.0.4.85/TBIOM.2022.3210479Published versio

Parsing Pointer Movements in a Target Unaware Environment

Analysis of the movements of the Mouse pointer could lead to valuable insights into a user’s mental status in digital environments. Previous research has yielded data showing a significant link between user mental status and pointer movements[1]. However, there is currently no standardized system to detect and parse out individual targeted movements of a mouse pointer by a user in an active environment. Active analysis of mouse movements could be useful in situations where the emotional state of the user is being measured. Data was collected through the Mathspring Project including results of problems solved, the facial expressions and self-reported emotions of students, and the movements of the mouse pointer, which is the focus of this work [3]. Although a connection has been shown in previous research [1], the ability to track this in a live system is held back by the manual process for splitting the motions of the pointer. The focus of this project is the development of a generalizable system to parse these movements automatically without needing much processing power or an immense amount of training data for each time. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG

Prediction of student engagement

A major challenge for online learning is the inability of systems to support student emotion and to maintain student engagement. In response to this challenge, computer vision has become an embedded feature in some instructional applications. In this paper, we propose a video dataset of college students solving math problems on the educational platform MathSpring.org with a front facing camera collecting visual feedback of student gestures. The video dataset is annotated to indicate whether students’ attention at specific frames is engaged or wandering. In addition, we train baselines for a computer vision module that determines the extent of student engagement during remote learning. Baselines include state-of-the-art deep learning image classifiers and traditional conditional and logistic regression for head pose estimation. We then incorporate a gaze baseline into the MathSpring learning platform, and we are evaluating its performance with the currently implemented approach.Published versio

Blinded by science?: Exploring affective meaning in students’ own words

This work addresses students’ open responses on causal attributions of their self-reported affective states. We use qualitative thematic data analysis techniques to develop a coding scheme by identifying common themes in students’ self-reported attributions. We then applied this scheme to a larger set of student reports. Analysis shows that students’ reasons for reporting a certain affect do not always align with researchers’ expectations. In particular, we discovered that a sizable group of students externalize their affect, attributing perceived difficulty of the problem and their own negativity as lying outside of themselves